Repository navigation
Add PR-description staleness guardrails (PROF-16592) - #980
Conversation
CI Test ResultsRun: #43440966751 | Commit:
Status Overview
Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled Summary: Total: 32 | Passed: 32 | Failed: 0 Updated: 2026-11-13 14:52:59 UTC |
Scan-Build Report
Bug Summary
Filter Results by FileReports
|
|||||||||||||||||||||||||||||||||||||||||||||||||
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b45dc07d8f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
On PRs with more than 100 commits, the ready-time guard can compare freshness against commit 100 instead of the head, allowing stale descriptions through.
🤖 Bits Code Review · Commit 5545c35
|
Agentic review of head The range changes What holds
High: the staleness check almost never fires (
Low
This is a comment only, not an approval or a change request. |
pre-push warns on every open-PR push to re-verify the description; the PR-ready subcommand blocks when the description predates the PR head commit, failing open; pr-writing skill and AGENTS.md route the re-verification.
CodeQL code-quality flags implicit adjacent string concatenation; same string, explicit +.
GitHub's updatedAt also moves on comments and pushed commits, so the ready gate allowed ready in exactly the stale-description case it guards. Compare lastEditedAt (createdAt fallback) against the head commit's date fetched via commits(last:1); commits returns only its first page, so .[-1] over it can miss the head.
Add the two tests the round-2 sphinx review flagged as mutation-escape gaps: a stale marker with a stale description must report the marker's reason (the returncode guard), and a failing check with no open PR must not block (the and-open_pr guard). Dedupe a doubled VIEW_NUMBER assignment.
b45dc07 to
8fd44f6
Compare
There was a problem hiding this comment.
PR URL targets can send the description-freshness probe to a different repository, allowing stale descriptions or blocking readiness based on an unrelated PR.
🤖 Bits Code Review · Commit 8fd44f6
The lastEditedAt probe reintroduced the implicit adjacent string concatenation the earlier fix removed from the old probe; same jq text, explicit +.
gh resolves a PR-URL selector's repository over -R, so a cross-repo ready command checked the -R repository's description. Match ready_target's resolution order. The test stub now answers only the queried owner/name and honors URL selectors, so a regression querying the wrong repository fails the suite.
gh pr view accepts URLs with trailing path fragments (/files) and still resolves them by the URL's repository; match the same prefix.
|
Addressed in the current head:
Deferred, with reasons: the two Two additional bot findings were fixed in this round: the graphql jq argument's implicit string concatenation (c5e344d) and the description probe querying the |
There was a problem hiding this comment.
More details
The PR adds guidance and push reminders to keep descriptions current, plus a freshness gate before marking PRs ready. The URL/repository mismatch is fixed, including URLs ending in /files, and no findings remain open; documented timestamp limitations and per-branch network costs remain.
🤖 Bits Code Review · Commit 9edb830
|
Agentic re-review of head Earlier findings
What holds
Medium: the description probe can read a different repository than
Low
This is a comment only, not an approval or a change request. |
rkennke
left a comment
There was a problem hiding this comment.
Looks good, feel free to fix the remaining minor issues here or in a follow-up, as you see fit.
rkennke re-review: the probe re-resolved the repository (http/GHES URLs and non-OWNER/NAME -R values read an unrelated PR or fail open, and gh repo view has no -R), and dated a head commit from a separate read. One node(id:) query now reuses the PR gh pr view resolved and pins the commit by the query's headRefOid against the marker-approved oid; a future committer date fails open. pre-push skips the open-PR probe for remote refs the push creates and defers the description warning until the push is decided. The test stub applies the command's real --jq expression and keys on the queried node id, so the jq path and wrong-PR regressions are exercised; the naive-timestamp test now proves fail-open.
Single quotes left $WT_TIP literal, so the test passed through the oid-mismatch guard instead of the empty-commits path it names.
|
Addressed in the current head:
Deferred, with reasons: the two One follow-up review finding was also fixed: the no-commits test's fixture left |
Reliability & Chaos Results✅ All reliability & chaos checks passed Pipeline: https://gitlab.ddbuild.io/DataDog/java-profiler/-/pipelines/152550713 |
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
3a40805
into
main
What does this PR do?:
Agents evolve a branch after its PR is open — review fixes, rebase replays, scope changes — without re-verifying the PR title and description. This adds a warn-level guardrail set that keeps descriptions in sync:
pr-writingskill rule plus anAGENTS.mdrouting line: whenever commits are added to a branch with an open PR, re-verify the title and description against the new diff and update what no longer holds..githooks/pre-push: every branch push that passes the marker, copyright and CLAUDE.md gates and targets a branch with an open PR warns (never blocks) to re-verify the description.gh pr viewprobe; comments also bumpupdatedAt, so a fresh value allows). The probe fails open.Motivation:
The pre-PR review gate certifies the code but nothing watches the description afterwards, so stale descriptions (wrong scope, dead numbers, dropped accepted costs) reach reviewers and merge. The pre-push path warns because the common write-description-then-commit-then-push flow has the description edit predating the head commit — a timestamp gate there would false-fire; the ready path blocks because marking ready is an explicit freshness claim.
Additional Notes:
Accepted costs, flagged for reviewer acknowledgment:
pre-pushnow makes onegh pr listcall on every branch push (previously skipped for a fresh marker); offline pushes fail open, at up to a 10s timeout.--force-with-leasereplay rewrites commit dates, so marking ready blocks until the description is re-saved — a no-op edit clears it.utils/README.mdknown limits).Agentic review (sphinx, seven rounds — initial, two fix re-reviews, post-rebase re-review, PR-thread fixes, and the follow-up re-review round): 0 CRITICAL/HIGH. LOW findings were fixed or dismissed with evidence (per-ref probe cost and probe fail-open are accepted design; git refnames cannot contain spaces, so the 4-token split in
ready_targetis safe; theisdigit/slash guards are behaviorally indistinguishable under fail-open). PR review threads are addressed: the description probe now dates the body by its own edit time, reads the head fromcommits(last: 1), and resolves the repository from the PR URL over-R.A base-commit spotless break (unformatted
JvmtiEventTimingTestfrom #947) was fixed in this branch and then dropped on rebase: #982 landed the identical formatting onmain. The gap it exposed — local gates cannot see a violation living in the base commit — remains unguarded.How to test the change?:
bash utils/tests/test_review_marker.sh— 273 tests, 0 failed (adds description-staleness block/equal/fresh, createdAt fallback, unparseable and naive timestamps, future-dated head, PR-moved fail-open, no-commits fail-open, pre-push warning deferral, created-remote-ref probe skip, marker-vs-description precedence, and cross-repository PR-URL description probes).For Datadog employees:
credentials of any kind, I've requested a security review (run the
dd:platform-security-reviewskill, or file a request via the PSEC review form).
bewairealso runs automatically on every PR.